Camera parameter derivation apparatus, camera parameter derivation method, and camera parameter derivation program

ABSTRACT

A camera parameter derivation apparatus for deriving camera parameters of a plurality of cameras for which conditions that the cameras be arranged at ideal positions that are set on a straight line at equal intervals and that directions of all arranged cameras be parallel to each other have been set includes an internal parameter derivation unit that derives internal parameter matrices of the cameras based on estimated internal parameter matrices for all cameras that are arranged at estimated positions while being oriented in estimated directions, a camera position derivation unit that derives ideal positions of the cameras that minimize the maximum of the distances between the estimated positions and the ideal positions, and a rotation matrix derivation unit that derives rotation matrices for correcting external parameters such that errors from parallel directions are equal to or less than a threshold value.

TECHNICAL FIELD

The present invention relates to a camera parameter derivationapparatus, a camera parameter derivation method, and a camera parameterderivation program.

BACKGROUND ART

Techniques for displaying an image that allows a subject to appear tostand out to the observer include a naked-eye stereoscopic display, alight field display, and the like. Multi-view images generated byphotographing a subject with multiple cameras are input to the lightfield display. The multi-view images of the subject are displayed on thelight field display as images of viewpoints, specifically, such that animage incident on the left eye of the observer and an image incident onthe right eye of the observer differ from each other. Using thedifference between images perceived by the left and right eyes, thelight field display can display an image of the subject that appears tostand out to the observer (see NPL 1).

The positions of the left and right eyes of the observer with respect tothe position of the light field display and the positions of two cameraswith respect to the position of the subject displayed on the light fielddisplay have the same positional relationship. Thus, it is necessary toaccurately install a plurality of cameras in a predetermined arrangementon the light field display. For example, an arrangement in which allcameras for photographing a subject are installed on a straight line atequal intervals and the directions of all cameras are parallel to eachother is predetermined as an ideal arrangement on the light fielddisplay disclosed in NPL 1.

However, it is difficult for an operator to install all cameras forphotographing a subject in an ideal arrangement. In the actualarrangement work, some of the cameras may not be installed on a straightline at equal intervals. Also, the direction of some of the cameras maynot be parallel to the directions of the remaining cameras.

In such cases, images to be projected to the left and right eyes of theobserver cannot be reproduced on the light field display with anaccurate positional relationship and thus an image that allows thesubject to appear three-dimensional to the observer cannot be accuratelyreproduced.

NPL 2 discloses a technique for correcting multi-view images generatedby a plurality of cameras not in an ideal arrangement into multi-viewimages generated by a plurality of cameras in an ideal arrangement. InNPL 2, camera parameters are derived for cameras in an ideal arrangementin which all cameras for photographing a subject are installed on astraight line at equal intervals and the directions of all cameras areparallel to each other. The multi-view images are corrected based on thederived camera parameters.

CITATION LIST Non Patent Literature

-   NPL 1: Munekazu Date, Megumi Isogai, Hideaki Kimata, “Full Parallax    Table Top Display Using Visually Equivalent Light Field 3D,”    Proceedings of the 23rd Annual Conference of the Virtual Reality    Society of Japan, Sep. 19-21, 2018, Tohoku University, Japan-   NPL 2: Jiachen Yang, Fei Guo, Huogen Wang, Zhiyong Ding, “A    Multi-View Image Rectification algorithm for matrix camera    arrangement,” Artificial Intelligence Research, 3(1), (2014): 18-29

SUMMARY OF THE INVENTION Technical Problem

Thus, when multi-view images are input to the light field display, it isnecessary to derive camera parameters for correcting the multi-viewimages with a certain level of accuracy or higher such that discomfortto an observer is reduced.

In view of the above circumstances, it is an object of the presentinvention to provide a camera parameter derivation apparatus, a cameraparameter derivation method, and a camera parameter derivation programcapable of deriving camera parameters for correcting multi-view imageswith a certain level of accuracy or higher such that discomfort to anobserver is reduced, when generating the multi-view images to bedisplayed on a light field display based on images taken by a pluralityof cameras.

Means for Solving the Problem

An aspect of the present invention provides a camera parameterderivation apparatus for deriving camera parameters of a plurality ofcameras for which conditions that the cameras be arranged at idealpositions that are set on a straight line at equal intervals and thatdirections of all arranged cameras be parallel to each other have beenset, the camera parameter derivation apparatus including an internalparameter derivation unit configured to derive, based on estimatedinternal parameter matrices for all cameras that are arranged atestimated positions that possibly have errors from the ideal positionswhile being oriented in estimated directions that possibly have errorsfrom the parallel directions, internal parameter matrices of the camerasfor correcting the estimated internal parameter matrices, a cameraposition derivation unit configured to derive, based on externalparameters of the arranged cameras, the ideal positions of the camerasthat minimize a maximum of distances between the estimated positions andthe ideal positions of the cameras, and a rotation matrix derivationunit configured to derive, based on the external parameters, rotationmatrices for correcting the external parameters such that the errorsfrom the parallel directions are equal to or less than a thresholdvalue.

Effects of the Invention

According to the present invention, it is possible to derive cameraparameters for correcting multi-view images with a certain level ofaccuracy or higher such that discomfort to an observer is reduced, whengenerating the multi-view images to be displayed on a light fielddisplay based on images taken by a plurality of cameras.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an exemplary configuration of a cameraparameter derivation apparatus according to a first embodiment.

FIG. 2 is a diagram illustrating an exemplary hardware configuration ofthe camera parameter derivation apparatus according to the firstembodiment.

FIG. 3 is a diagram illustrating an exemplary arrangement of a pluralityof cameras according to the first embodiment.

FIG. 4 is a flowchart showing an exemplary operation of the cameraparameter derivation apparatus according to the first embodiment.

FIG. 5 is a flowchart showing an exemplary operation of deriving areference position and a distance vector according to the firstembodiment.

FIG. 6 is a diagram illustrating an exemplary configuration of a cameraparameter derivation apparatus according to a second embodiment.

FIG. 7 is a flowchart showing an exemplary operation of the cameraparameter derivation apparatus according to the second embodiment.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will be described in detail withreference to the drawings.

First Embodiment

FIG. 1 is a diagram illustrating an exemplary configuration of a cameraparameter derivation apparatus 1 a. The camera parameter derivationapparatus 1 a is an apparatus for deriving camera parameters for aplurality of cameras. The camera parameter derivation apparatus 1 aderives ideal camera parameters (virtual camera parameters) forcorrecting multi-view images with a certain level of accuracy or highersuch that discomfort to an observer is reduced, when generating themulti-view images to be displayed on a light field display based onimages taken by a plurality of cameras. The camera parameter derivationapparatus 1 a derives ideal internal parameters of each camera, an idealposition of each camera, and an ideal rotation matrix of each camera.

An ideal arrangement is an arrangement that satisfies predeterminedconditions. In the following, the predetermined conditions areconditions that all cameras for photographing a subject be installed ona straight line at equal intervals with predetermined accuracy and thatthe directions of all cameras be parallel to each other withpredetermined accuracy.

Camera parameters include external parameters and internal parameters.The external parameters are parameters representing the position anddirection (orientation) of the camera, examples of which are parametersrepresenting a translation vector (parallel translation vector)representing the position of the camera and a rotation matrixrepresenting the orientation of the camera. The internal parameters areparameters for converting camera coordinates into image coordinates,examples of which are a parameter representing a focal length, aparameter representing the position of an optical center (principalpoint), and a parameter representing lens distortion.

The camera parameter derivation apparatus 1 a includes an internalparameter derivation unit 10, a camera position derivation unit 11 a,and a rotation matrix derivation unit 12 a. The camera positionderivation unit 11 a includes a temporary position derivation unit 110and an ideal position derivation unit 111 a.

FIG. 2 is a diagram illustrating an exemplary hardware configuration ofthe camera parameter derivation apparatus 1 a. The camera parameterderivation apparatus 1 a includes a processor 100, a storage unit 200,and a communication unit 300.

Some or all of the internal parameter derivation unit 10, the cameraposition derivation unit 11 a, and the rotation matrix derivation unit12 a are implemented in software by the processor 100 such as a centralprocessing unit (CPU) executing a program stored in a storage unit 200having a non-volatile recording medium (non-temporary recording medium).The program may be recorded on a computer-readable recording medium. Thecomputer-readable recording medium is a portable medium such as aflexible disk, a magneto-optical disc, a read only memory (ROM), or aCD-ROM or a non-temporary recording medium such as a storage device suchas a hard disk provided in a computer system. The program may bereceived by the communication unit 300 via a communication line. Thestorage unit 200 may store the camera parameters of each camera, imagedata, data tables, and lookup tables.

Some or all of the internal parameter derivation unit 10, the cameraposition derivation unit 11 a, and the rotation matrix derivation unit12 a may be implemented, for example, using hardware including anelectrical circuit or circuitry that uses a large scale integration(LSI) circuit, an application specific integrated circuit (ASIC), aprogrammable logic device (PLD), or a field programmable gate array(FPGA).

Hereinafter, a position estimated as an actual position of a camera isreferred to as an “estimated position.” An ideal position of a camera isreferred to as an “ideal position.” A temporary position as an idealposition of a camera is referred to as a “temporary ideal position.” Aposition that serves as a reference for the positions of cameras isreferred to as a “reference position.” A temporary position as thereference position for cameras is referred to as a “temporary referenceposition.” An ideal distance between adjacent cameras is referred to asan “ideal distance.” A temporary distance as the ideal distance betweencameras is referred to as a “temporary ideal distance.” A vector of thedistance between adjacent ideal positions is referred to as an “idealdistance vector.” A vector of the distance between adjacent temporarypositions is referred to as a “temporary ideal distance vector.”

FIG. 3 is a diagram illustrating an exemplary arrangement of a pluralityof cameras 2-i (where i=1, . . . , n). The positions of the arranged ncameras 2-i “C_(i)” are estimated positions of the cameras. Estimatedpositions may have errors from ideal positions. The number of cameras2-i “n” represents the number of viewpoints of multi-view images inputto the light field display.

Hereinafter, “d^(ideal)” represents the ideal distance vector. “C_(i)^(ideal)” represents the ideal position of each camera 2-i. “C_(i)^(real)” represents the estimated position of each camera 2-i “C_(i).”The ideal position of the camera 2-1 “C₁” is defined as the referenceposition.

The n cameras 2-i photograph a subject. Each camera 2-i generates amoving image or a still image including an image of a subject (notillustrated). Thus, multi-view images are generated from the imagestaken by the n cameras 2-i. The internal and external parameters of eachcamera 2-i are estimated in advance, for example, by a calibrationmethod disclosed in Reference 1.

-   Reference 1: Zhang Zhengyou, “A flexible new technique for camera    calibration,” IEEE Transactions on Pattern Analysis and Machine    Intelligence, 22(11), (2000): 1330-1334.

Referring back to FIG. 1 , an outline of the camera parameter derivationapparatus 1 a will be described.

The internal parameter derivation unit 10 derives and outputs an idealinternal parameter matrix of each camera 2-i based on the inputestimated internal parameters. The temporary position derivation unit110 derives and outputs a temporary reference position for each camera2-i based on the input estimated external parameters. The temporaryposition derivation unit 110 derives and outputs a temporary idealdistance vector for the cameras 2-i based on the input estimatedexternal parameters. The ideal position derivation unit 111 a derives anideal position of each camera 2-i based on the temporary referenceposition and the temporary ideal distance vector. The ideal positionderivation unit 111 a derives an ideal distance vector for the n cameras2-i and outputs the ideal distance vector to the rotation matrixderivation unit 12 a. The rotation matrix derivation unit 12 a derivesan ideal rotation matrix of each camera 2-i based on the estimatedexternal parameters and the ideal distance vector for the n cameras 2-i.

Next, details of the camera parameter derivation apparatus 1 a will bedescribed. The internal parameter derivation unit 10 takes internalparameters estimated in previously executed calibration as inputs andderives and outputs an ideal internal parameter matrix. A method bywhich the internal parameter derivation unit 10 derives the idealinternal parameter matrix is not limited to a specific derivationmethod.

For example, the internal parameter derivation unit 10 may derive anaverage of the internal parameter matrices of all cameras 2-i as anideal internal parameter matrix. For example, the internal parameterderivation unit 10 may derive an average of the internal parametermatrices of all cameras 2-i and derive an internal parameter matrixclosest to the average as an ideal internal parameter matrix. Forexample, the internal parameter derivation unit 10 may derive aninternal parameter matrix of a camera 2-i selected from all arrangedcameras 2-i as an ideal internal parameter matrix.

The camera position derivation unit 11 a acquires external parametersestimated in the previously executed calibration. The camera positionderivation unit 11 a takes the estimated external parameters as inputsand derives and outputs a reference position “C₁ ^(ideal)” and an idealdistance vector “d^(ideal)” between adjacent ideal positions “C_(i)^(ideal).” The camera position derivation unit 11 a derives and outputsan ideal position “C_(i) ^(ideal)” of each camera 2-i based on thereference position “C_(i) ^(ideal)” and the ideal distance vector“d^(ideal)” between adjacent ideal positions “C_(i) ^(ideal).”

In order for the multi-view images input to the light field display tobe corrected with a certain level of accuracy or higher such thatdiscomfort to an observer is reduced, it is desirable that all cameras2-i be arranged in the same directions at positions that are on astraight line at equal intervals and the distances between the estimatedpositions and the ideal positions be short. Therefore, the cameraposition derivation unit 11 a derives expression (1) to minimize themaximum of the distances (errors) between the estimated positions andthe ideal positions.

[Math. 1]

minimize max{D ₁ ,D ₂ , . . . ,D _(n)}  (1)

Here, “D_(i)” represents the distance between the estimated position“C_(i) ^(real)” and the ideal position “C_(i) ^(ideal)” of each camera2-i. The distance between the estimated position “C_(i) ^(real)” and theideal position “C_(i) ^(ideal)” of each camera 2-i is represented byequation (2).

[Math. 2]

D _(i) =∥C _(i) ^(real) −C _(i) ^(ideal)∥₂  (2)

Here, the ideal position “C_(i) ^(ideal)” is represented by equation (3)using the ideal distance vector “d^(ideal).”

[Math. 3]

C _(i) ^(ideal) =C _(i) ^(ideal)+(i−1)d ^(ideal)  (3)

The camera position derivation unit 11 a derives the reference position“C₁ ^(ideal)” and the ideal distance vector “d^(ideal)” by solvingexpression (1). A method by which the camera position derivation unit 11a solves expression (1) is not limited to a specific method. The cameraposition derivation unit 11 a may derive the reference position “C_(i)^(ideal)” and the ideal distance vector “d^(ideal),” for example, byperforming a full search.

Next, a method of deriving the reference position “C₁ ^(ideal)” and theideal distance vector “d^(ideal)” will be described.

The temporary position derivation unit 110 takes the external parametersestimated in the previously executed calibration as inputs, derives atemporary reference position “C′₁” and a temporary ideal distance vector“d′,” and outputs the derived temporary reference position “C′₁” andtemporary ideal distance vector “d” to the ideal position derivationunit 111 a. Here, the temporary position derivation unit 110 derives thetemporary reference position “C′₁” and the temporary ideal distancevector “d′,” by solving expression (4) that minimizes the sum of thedistances (errors) between the estimated positions “C_(i) ^(real)” andthe temporary ideal positions “C′_(i).” The temporary positionderivation unit 110 outputs the temporary reference position “C′₁” andthe temporary ideal distance “d” to the ideal position derivation unit111 a.

$\begin{matrix}\left\lbrack {{Math}.4} \right\rbrack &  \\{{minimize}{\sum\limits_{i = 1}^{n}{{C_{i}^{real} - C_{i}^{\prime}}}_{2}}} & (4)\end{matrix}$

Here, “C′_(i)” represents the temporary ideal position of each camera2-i. “C′_(i)” is represented by equation (5) using the temporaryreference position “C′₁” and the temporary ideal distance vector “d′.”

[Math. 5]

C ₁ ′=C ₁′+(i−1)d′  (5)

The ideal position derivation unit 111 a takes the derived temporaryreference position “C′₁” and temporary ideal distance vector “d” asinputs and sets them as initial values for expression (1). The idealposition derivation unit 111 a derives the reference position “C₁^(ideal)” of the camera 2-1 and the ideal distance vector “d^(ideal)” byadjusting the temporary reference position “C′₁” and the temporary idealdistance vector “d” within a predetermined range and outputs the derivedreference position “C₁ ^(ideal)” and ideal distance vector “d^(ideal).”

The range of adjustment is not limited to a specific range. For example,the ideal position derivation unit 111 a may set the distance from thetemporary reference position “C′₁” to the closest estimated position “C₂^(real)” as an upper limit of the range in which the temporary referenceposition “C′₁” is to be moved. The ideal position derivation unit 111 amay also set an average of the distances between adjacent estimatedpositions “C_(i) ^(real)” as a distance to be added or subtracted fromthe temporary ideal distance vector “d′.”

The ideal position derivation unit 111 a derives and outputs the idealposition “C_(i) ^(ideal)” of each camera 2-i based on the referenceposition “C₁ ^(ideal)” of the camera 2-1 and the ideal distance vector“d^(ideal)” as in equation (2).

The rotation matrix derivation unit 12 a acquires the externalparameters estimated in the previously executed calibration and takesthem as inputs. The rotation matrix derivation unit 12 a acquires theideal distance vector “d^(ideal)” for the n cameras 2-i from the idealposition derivation unit 111 a. Thereby, the rotation matrix derivationunit 12 a can derive ideal rotation matrices for making the directionsof all cameras 2-i parallel to each other. Based on the ideal internalparameter matrix, the ideal positions, and the ideal rotation matrices,the images taken by the cameras 2-i are converted into images taken bythe cameras 2-i at the ideal positions, thereby generating multi-viewimages that can be accurately reproduced by the light field display.

A method for deriving the ideal rotation matrix is not limited to aspecific method. For example, the rotation matrix derivation unit 12 amay use the unit vector of the ideal distance vector as the x componentof the ideal rotation matrix. Here, the rotation matrix derivation unit12 a uses the outer product of the z component of a rotation matrix fora camera 2-i selected from the arranged cameras 2-i and the x componentof the ideal rotation matrix as the y component of the ideal rotationmatrix. The rotation matrix derivation unit 12 a may use the outerproduct of the x component and they component of the ideal rotationmatrix as the z component of the ideal rotation matrix.

Next, an exemplary operation of the camera parameter derivationapparatus 1 a will be described.

FIG. 4 is a flowchart showing an exemplary operation of the cameraparameter derivation apparatus 1 a. The internal parameter derivationunit 10 acquires estimated internal parameters. The internal parameterderivation unit 10 takes the estimated internal parameters as inputs andderives an ideal internal parameter matrix. The internal parameterderivation unit 10 outputs the ideal internal parameter matrix (stepS101). Step S101 may be executed after any of steps S102 to S104 hasbeen executed.

The camera position derivation unit 11 a acquires estimated externalparameters. The camera position derivation unit 11 a takes the estimatedexternal parameters as inputs and derives and outputs a referenceposition “C₁ ^(ideal)” and an ideal distance vector “d^(ideal)” betweenadjacent ideal positions “C_(i) ^(ideai)” (step S102).

The camera position derivation unit 11 a derives and outputs an idealposition “C_(i) ^(ideal)” of each camera 2-i based on the referenceposition “C₁ ^(ideal)” and the ideal distance vector “d^(ideal)” betweenadjacent ideal positions “C_(i) ^(ideal)” (step S103).

The rotation matrix derivation unit 12 a acquires the estimated externalparameters. The rotation matrix derivation unit 12 a acquires the idealdistance vector “d^(ideal)” between adjacent ideal positions “C_(i)^(ideal)” from the camera position derivation unit 11 a. The rotationmatrix derivation unit 12 a derives and outputs an ideal rotation matrixof each camera 2-i based on the estimated external parameters and theideal distance vector “d^(ideal)” between adjacent ideal positions“C_(i) ^(ideal)” (step S104).

FIG. 5 is a flowchart showing an exemplary operation of deriving thereference position “C₁ ^(ideal)” and the distance vector “d^(ideal).”The temporary position derivation unit 110 acquires the estimatedexternal parameters. The temporary position derivation unit 110 takesthe estimated external parameters as inputs and derives the sum of thedistances between the estimated positions “C_(i) ^(real)” and thetemporary ideal positions “C′_(i).” The temporary position derivationunit 110 derives and outputs a temporary reference position “C′1” and atemporary ideal distance vector “d” between adjacent temporary idealpositions based on the sum of the distances between the estimatedpositions “C_(i) ^(real)” and the temporary ideal positions “C′_(i)”(for example, based on expression (4)) (step S201).

The ideal position derivation unit 111 a acquires the temporaryreference position “C′₁” and the ideal distance vector “d” betweenadjacent temporary ideal positions from the temporary positionderivation unit 110. The ideal position derivation unit 111 a takes thetemporary reference position “C′₁” and the temporary ideal distancevector “d′” between adjacent temporary ideal positions as inputs andderives a reference position “C₁ ^(ideal)” and an ideal distance vector“d^(ideal)” between adjacent ideal positions. The ideal positionderivation unit 111 a derives an ideal position of each camera 2-i basedon the reference position “C₁ ^(ideal)” and the ideal distance vector“d^(ideal)” between adjacent ideal positions. The ideal positionderivation unit 111 a outputs the ideal position of each camera 2-i toan external device (not illustrated). The ideal position derivation unit111 a outputs the ideal distance vector “d^(ideal)” between adjacentideal positions to the rotation matrix derivation unit 12 a (step S202).

The camera parameter derivation apparatus 1 a derives camera parameters(internal parameter matrices and external parameters) of a plurality ofcameras 2-i for which conditions that the cameras 2-i be arranged atideal positions that are set on a straight line at equal intervals andthat the directions of all arranged cameras 2-i be parallel to eachother have been set as described above. The cameras 2-1 to 2-n arearranged at estimated positions that may have errors from the idealpositions while being oriented in estimated directions that may haveerrors from the parallel directions.

Based on estimated internal parameter matrices for all cameras 2-i thatare arranged at the estimated positions while being oriented in theestimated directions that may have errors from parallel directions, theinternal parameter derivation unit 10 derives internal parametermatrices of the cameras 2-i for correcting the estimated internalparameter matrices.

Based on the external parameters of the arranged cameras 2-i, the cameraposition derivation unit 11 a derives ideal positions of the cameras 2-iwhich minimize the maximum of the distances “D_(i)” between theestimated positions “C_(i) ^(real)” and the ideal positions “C_(i)^(ideal)” of the cameras 2-i (for example, expression (1)). The cameraposition derivation unit 11 a may derive a temporary reference positionand a distance vector between temporary ideal positions such that thesum of the distances between the estimated positions and the temporaryideal positions is minimized (for example, expression (4)). The cameraposition derivation unit 11 a may derive ideal positions that minimizethe maximum of the distances between the estimated positions and theideal positions by adjusting the temporary reference position and thedistance vector between temporary ideal positions. Based on the externalparameters, the rotation matrix derivation unit 12 a derives rotationmatrices for correcting the external parameters such that the errorsfrom the parallel directions are equal to or less than a thresholdvalue. The rotation matrix derivation unit 12 a derives rotationmatrices based on the external parameters and the distance vectorbetween adjacent ideal positions.

Based on the external parameters of the arranged cameras 2-i, the cameraposition derivation unit 11 a derives and outputs ideal positions of thecameras 2-i that minimize the maximum of the distances between theestimated positions and the ideal positions of the cameras 2-i asdescribed above. Thereby, it is possible to derive camera parameters forcorrecting multi-view images with a certain level of accuracy or highersuch that discomfort to an observer is reduced, when generating themulti-view images to be displayed on a light field display based onimages taken by a plurality of cameras. It is also possible toefficiently derive the ideal position of each camera which is one of thecamera parameters with a reduced amount of computation.

Second Embodiment

A second embodiment differs from the first embodiment in that a rotationmatrix derivation unit derives ideal rotation matrices based only onexternal parameters. In the second embodiment, differences from thefirst embodiment will be mainly described.

FIG. 6 is a diagram illustrating an exemplary configuration of a cameraparameter derivation apparatus 1 b. The camera parameter derivationapparatus 1 b includes an internal parameter derivation unit 10, acamera position derivation unit 11 b, and a rotation matrix derivationunit 12 b. The camera position derivation unit 11 b includes a temporaryposition derivation unit 110 and an ideal position derivation unit 111b.

The ideal position derivation unit 111 b acquires a temporary referenceposition “C′ i” and a distance vector “d” between adjacent temporaryideal positions from the temporary position derivation unit 110. Theideal position derivation unit 111 b takes the temporary referenceposition “C′₁” and the distance vector “d” between adjacent temporaryideal positions (for example, those of equation (5)) as inputs andderives a reference position “C₁ ^(ideal)” and a distance vector“d^(ideal)” between adjacent ideal positions. The ideal positionderivation unit 111 b derives an ideal position of each camera 2-i basedon the reference position “C_(i) ^(ideal)” and the distance vector“d^(ideal)” between adjacent ideal positions. The ideal positionderivation unit 111 b outputs the ideal position of each camera 2-i toan external device (not illustrated).

The rotation matrix derivation unit 12 b acquires external parametersestimated in previously executed calibration. The rotation matrixderivation unit 12 b takes the estimated external parameters as inputsand derives an ideal rotation matrix of each camera 2-i. The rotationmatrix derivation unit 12 a outputs ideal rotation matrices for makingthe directions of all cameras 2-i parallel to each other to an externaldevice (not illustrated). A method by which the rotation matrixderivation unit 12 b derives the ideal rotation matrices is not limitedto a specific method. For example, the rotation matrix derivation unit12 b may derive an average of the rotation matrices of the cameras as anideal rotation matrix.

Next, an exemplary operation of the camera parameter derivationapparatus 1 b will be described.

FIG. 7 is a flowchart showing an exemplary operation of the cameraparameter derivation apparatus 1 b. Steps S301 to S303 are the same assteps S101 to S103 shown in FIG. 4 .

The rotation matrix derivation unit 12 b acquires estimated externalparameters. The rotation matrix derivation unit 12 b takes the estimatedexternal parameters as inputs and derives and outputs ideal rotationmatrices of the cameras 2-i for making the directions of all cameras 2-iparallel to each other (step S304). The execution order of steps S301and S302 does not matter in deriving the ideal rotation matrices in stepS304.

In the camera parameter derivation apparatus 1 b, the rotation matrixderivation unit 12 b derives the rotation matrices based on the externalparameters as described above. Thereby, it is possible to derive cameraparameters for correcting multi-view images such that discomfort to anobserver is reduced, when generating the multi-view images to bedisplayed on a light field display based on images taken by a pluralityof cameras.

Although embodiments of the present invention have been described abovein detail with reference to the drawings, the specific configurationsthereof are not limited to those of the embodiments and also includedesigns or the like without departing from the spirit of the presentinvention.

INDUSTRIAL APPLICABILITY

The present invention can be applied to an image processing apparatusthat derives camera parameters for correcting multi-view imagesdisplayed on a light field display or the like.

REFERENCE SIGNS LIST

-   -   1 a, 1 b Camera parameter derivation apparatus    -   2 Camera    -   10 Internal parameter derivation unit    -   11 a, 11 b Camera position derivation unit    -   12 a, 12 b Rotation matrix derivation unit    -   20 Estimated position    -   21 Ideal position    -   100 Processor    -   110 Temporary position derivation unit    -   111 a, 111 b Ideal position derivation unit    -   200 Storage unit    -   300 Communication unit

1. A camera parameter derivation apparatus for deriving cameraparameters of a plurality of cameras for which conditions that thecameras be arranged at ideal positions that are set on a straight lineat equal intervals and that directions of all arranged cameras beparallel to each other have been set, the camera parameter derivationapparatus comprising: a processor; and a storage medium having computerprogram instructions stored thereon, when executed by the processor,perform to: derive, based on estimated internal parameter matrices forall cameras that are arranged at estimated positions that possibly haveerrors from the ideal positions while being oriented in estimateddirections that possibly have errors from the parallel directions,internal parameter matrices of the cameras for correcting the estimatedinternal parameter matrices; derive, based on external parameters of thearranged cameras, the ideal positions of the cameras that minimize amaximum of distances between the estimated positions and the idealpositions of the cameras; and derive, based on the external parameters,rotation matrices for correcting the external parameters such that theerrors from the parallel directions are equal to or less than athreshold value.
 2. The camera parameter derivation apparatus accordingto claim 1, wherein the computer program instructions further perform toderive a temporary reference position and a distance vector between theadjacent temporary ideal positions such that a sum of distances betweenthe estimated positions and the temporary ideal positions is minimizedand to derive the ideal positions that minimize the maximum by adjustingthe temporary reference position and the distance vector between theadjacent temporary ideal positions.
 3. The camera parameter derivationapparatus according to claim 1, wherein the rotation matrix derivationunit is configured to derive the rotation matrices based on the externalparameters and a distance vector between the adjacent ideal positions.4. A camera parameter derivation method performed by a camera parameterderivation apparatus for deriving camera parameters of a plurality ofcameras for which conditions that the cameras be arranged at idealpositions that are set on a straight line at equal intervals and thatdirections of all arranged cameras be parallel to each other have beenset, the camera parameter derivation method comprising: deriving, basedon estimated internal parameter matrices for all cameras that arearranged at estimated positions that possibly have errors from the idealpositions while being oriented in estimated directions that possiblyhave errors from the parallel directions, internal parameter matrices ofthe cameras for correcting the estimated internal parameter matrices;deriving, based on external parameters of the arranged cameras, theideal positions of the cameras that minimize a maximum of distancesbetween the estimated positions and the ideal positions of the cameras;and deriving, based on the external parameters, rotation matrices forcorrecting the external parameters such that the errors from theparallel directions are equal to or less than a threshold value.
 5. Anon-transitory computer-readable medium having computer-executableinstructions that, upon execution of the instructions by a processor ofa computer, cause the computer to function as the camera parameterderivation apparatus according to claim 1.